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10 BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

This invention relates generally to computer generated images and more 
particularly to a system and method for environment mapping. 



15 2. Description of the Background Art 

Typically, the illumination of a computer-generated object by discrete 
light sources, continuous light sources, and ambient light is described by an 
illumination model. The object is illuminated by the reflection of ambient light 
and the reflection of light source light from the surface of the object. Generally, 

20 the illumination model is a mathematical expression that operates on a set of 
variables to generate reflection properties, such as color and intensity of 
reflected light and an object's texture as viewed by an observer. Given ambient 
light and light sources positioned about the object, the illumination model 
defines the reflection properties of the object. The illumination model is 

25 considered to be accurate if the illuminated object appears realistic to an 
observer. 

Typically, the illumination model is incorporated in a software program 
executed by a vector processing unit, a central processing unit, or a rendering 
engine of a computer system. The program must be capable of computing the 
30 illumination of the object when the light sources change position with respect 
to the object, when the observer views the illuminated object from a different 
angle, or when the object is rotated. Furthermore, an efficient illumination 
model is needed for the processing unit to compute the illumination in real- 
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time, for example, if the observer (i.e., a camera) is moving with respect to the 
object. Therefore, it is desired to incorporate terms in the illumination model 
that are computationally cost effective, while at the same time generating an 
image of the illuminated object that is aesthetically pleasing to the observer. 
5 Computing texture (i.e., environment mapping) is important when 

rendering a realistic image of the illuminated object that closely resembles a 
real physical object. Typically, texture coordinates for each point of the object's 
surface are computed, and a texture map comprising the texture coordinates is 
generated. 

10 FIG. 1 illustrates a prior art direct normal projection method for 

computing an object's texture coordinates. FIG. 1 includes an object's surface 
105, a point P on surface 105, a normal vector n to surface 105 at point P, an 
observer 1 10a, a line-of sight 1 15a between observer 1 10a and the point P, and 
a projection of the normal vector n onto an x-axis 120, referred to as n x . In 

15 general, a z-axis (not shown) is perpendicular to x-axis 120 and is in the plane 
of FIG. 1, and a y-axis (not shown) is perpendicular to x-axis 120 and the z-axis 
and is out of the plane of FIG. 1. For simplicity of illustration, the FIG. 1 
embodiment of object's surface 105 is a line, however, surface 105 is typically 
any 2-D surface, and hence in general, the normal vector n may have a vector 

20 component n y along the y-axis. 

In operation, the direct normal projection method computes the projected 
components n x and n y of the normal vector n for each point P on object's 
surface 105. The central processing unit or vector processing unit then maps 
(i.e., transforms) the projected components n x and n y into texture coordinates 

25 (s,t) using one or more mapping algorithms known in the* art. The vector 

processing unit then uses the computed texture coordinates (s,t) for each point 
P, as well as other reflection variables, in an illumination model to generate a 
reflection pattern of object's surface 105. Although the direct normal projection 
method of the prior art may be fast, the method generates a reflection pattern 

30 that appears "painted-on" as observer 110a moves to different locations. In 

other words, the reflection pattern of object's surface 105 does not change with 

♦ 

respect to rotation or translation of observer 1 10a, since the method depends 
upon the x and y components of the normal vector n, independent of the 
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position of observer 1 10a with respect to the point P. For example, the vector 
processing unit computes the same projected components (n x ,%) and texture 
coordinates (s,t) for an observer 1 10b viewing point P as observer 1 10a viewing 
point P. 

It would be useful to implement a system and method of environment 
mapping that depends upon an observer's location with respect to an object's 
location and orientation to generate a more realistic reflection pattern, and that 
is consistent with results of the direct normal projection method for particular 
object-observer geometries. 
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SUMMARY OF THE INVENTION 
In accordance with the present invention, a system and method for 
environment mapping of a reflective object is disclosed. In one embodiment of 
the invention, the method includes constructing a surface normal vector n at a 
5 point P on a surface of the reflective object, constructing an observation vector 
e from the point P to an observer, and using a modified reflection formula to 
compute a reflection vector r based on the surface normalvector n and the 
observation vector e. The modified reflection formula is based on reflection 
about a pseudo-normal vector n' at the point P on the surface. 

10 According to the present invention, the pseudo-normal vector n' bisects 

an angle subtended by the surface normal vector n and a reference observation 
vector e 0 , where the reference observation vector e 0 is directed from the point P 
to an observer located directly in front of the point P. 
The modified reflection formula is: 

15 r = e - (e-(n+e 0 ))(n+e 0 )/(l-nz) = e - (e ; [nx, ny, nz-l])[nx, ny, nz-l]/(l - nz), where 
e 0 = [0,0,-1], and nx, ny, and nz are the components of the surface normal 
vector n. Each computed reflection vector r may be processed to generate a 
pair of texture coordinates (s,t). The reflective object is then rendered based in 
part on the texture coordinates (s,t) associated with each point P on the surface 

20 of the reflective object. The scope of the present invention covers all types of 
rendering schemes, such as a polygon rendering where each point P on the 
surface of the reflective object is located at the vertex of a polygon. 

In another embodiment of the invention, the system includes a memory 
configured to store a modified reflection model, a vector processing unit 

25 configured to compute reflection vectors using the modified reflection model, 
and a graphics processor configured to render the reflective object in an image. 
The quality of the image is dependent upon the texture coordinates that are 
derived from the computed reflection vectors. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
FIG. 1 illustrates a prior art direct normal projection method for 
computing an object's texture coordinates; 

FIG. 2 is a block diagram of one embodiment of an electronic 
5 entertainment system according to the invention; 

FIG. 3 illustrates a modified reflection projection method stored in main 
memory 210 of FIG. 2, according to one embodiment of the invention; 

FIG. 4 is a flowchart of method steps for displaying an image of a 
reflective object based upon texture coordinates, according to one embodiment 
10 of the invention; and 

FIG. 5 is a flowchart of method steps for step 415 of FIG. 4 to generate 
reflection vectors for a reflective object, according to one embodiment of the 
invention. 
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DETAILED DESCRIPTION OF THE INVENTION 
The system and method for environment mapping described herein allow 
a computer-generated object's reflective appearance to change, based upon 
position and orientation of a camera with respect to the object's location. A 
5 position of the camera may be defined by a lateral location of the camera with 
respect to the object's location. Lateral camera movement is defined as motion 
to the right, left, up, or down with respect to the object's location. Camera 
orientation may be defined by rotation angles with respect to a given, fixed 
coordinate system. 

10 An exemplary embodiment of the invention is implemented as a real-time 

environment mapping for polygon rendering. However; the scope of the 
invention covers other applications, such as environment mapping for other 
rendering schemes. Other rendering schemes may include, but are not limited 
to, point-based and non-polygon volume-based primitives. Various 

15 embodiments of the invention may be enabled in software, hardware, or 
firmware . 

According to one embodiment of the invention, a central processing 
unit (CPU) and/or one or more vector processing units (VPUs) use 
illumination models to compute reflective properties of an object. The 

20 object's reflective properties are associated with the objects' appearance. 
Reflective properties include color and intensity of light reflected by the 
object, and texture of the reflective object. The texture of an object is 
associated with reflective properties such as the object's shininess and 
overall surface appearance. Typically, the object's texture is specified by 

25 texture coordinates (s,t) computed by the VPU. Texture coordinates may be 
incorporated into a texture map which is wrapped (i.e., mapped) around the 
object. For example, a VPU may execute environment mapping instructions 
that operate on variables stored in a VPU random access memory (RAM) or 
on variables stored in a CPU register to compute the texture coordinates. 

30 Typically the texture coordinates and the other computed reflective 

properties (also referred to as illumination terms) such as color and intensity 
are passed to a graphics processing unit (GPU) for further processing. 
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Subsequently, the GPU prepares the reflective object for display on a display 
device such as a computer monitor. 

FIG. 2 is a block diagram of one embodiment of an electronic 
entertainment system 200 according to the invention. System 200 includes, 
5 but is not limited to, a main memory 210, a CPU 212, a VPU 213, a GPU 214, 
an input/output processor (IOP) 216, an IOP memory 218, a controller 
interface 220, a memory card 222, a Universal Serial Bus^USB) interface 224, 
and an IEEE 1394 interface 226. System 200 also includes an operating 
system read-only memory (OS ROM) 228, a sound processing unit (SPU) 232, 

10 an optical disc control unit 234, and a hard disc drive (HDD) 236, which are 
connected via a bus 246 to IOP 216. 

CPU 212, VPU 213, GPU 214, arid IOP 216 communicate via a system 
bus 244. CPU 212 communicates with main memory 210 via a dedicated bus 
242. VPU 213 and GPU 214 may also communicate via a dedicated bus 240. 

15 CPU 212 executes programs stored in OS ROM 228 and main memory 

210. Main memory 210 may contain pre-stored programs and may also 
contain programs transferred via IOP 216 from a CD-ROM or DVD-ROM (not 
shown) using optical disc control unit 234. IOP 216 controls data exchanges 
between CPU 212, VPU 213, GPU 214 and other devices of system 200, such as 

20 controller interface 220. 

Main memory 210 includes, but is not limited to, a program having game 
instructions including an illumination model. The program is preferably loaded 
from a DVD-ROM via optical disc control unit 234 into main memory 210. CPU 

212, in conjunction with VPU 213, GPU 214, and SPU 232, executes game 

■ 

25 instructions and generates rendering instructions using inputs received from a 
user via controller interface 220. The user may also instruct CPU 212 to store 
certain game information on memory card 222. Other devices may be 
connected to system 200 via USB interface 224 and IEEE 1394 interface 226. 
In one embodiment of the invention, VPU 213 executes instructions from 

30 CPU 212 to generate texture coordinates associated with an illuminated object 
by using the mumination model. SPU 232 executes instructions from CPU 212 
to produce sound signals that are output on an audio device (not shown). GPU 
214 executes rendering instructions from CPU 212 and VPU 213 to produce 
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images for display on a display device (.not shown). That is, GPU 214, using the 
texture coordinates and other illumination terms generated by VPU 213, and 
rendering instructions from CPU 212, renders the illuminated object in an 
image. 

5 FIG. 3 illustrates a modified environment reflection projection method 

stored in main memory 210 of FIG. 2, according to the present invention. In 
one embodiment of the invention, the modified reflection projection method is 
used by the illumination model. In another embodiment of the invention, the 
modified reflection projection method is incorporated into the iUumination 

10 model. FIG. 3 includes an object's surface 305, a point P on the surface 305, a 
normal vector n to the surface 305 at point P (also referred to as a surface 
normal vector), an observer 310a, an observation vector e directed from the 
point P to observer 310a, and an x-axis 320. In one embodiment of the 
invention, surface 305 is composed of polygon primitives (not shown), and at 

15 each vertex of each polygon, a point P is specified on surface 305. For future 
reference, the z-axis (not shown) is perpendicular to x-axis 320 and is in the 
plane of FIG. 3, and the y-axis (not shown) is perpendicular to x-axis 320 and 
the z-axis and is out of the plane of FIG. 3. For simplicity of illustration, the 
FIG. 3 embodiment of surface 305 is a line, however, any point P on any two- 

20 dimensional surface is within the scope of the invention. For example, the FIG. 
3 embodiment of surface 305 may be the intersection of a two-dimensional 
surface (not shown) with the x-z plane, and thus the normal vector n may have 
a vector component n y along the y-axis. 

•* 

According to one embodiment of the invention, the modified environment 
25 reflection projection method uses a modified version of a standard reflection 
formula to calculate a reflection vector r for each point P on surface 305. The 
method then processes the reflection vfector r to generate texture coordinates 
(s,t) for each point P. The standard reflection formula is r = e - 2(e-n)n. For 
each point P specified on surface 305 with a given normal n, the standard 
30 reflection formula gives a reflection vector r based upon a given observer 

position specified by the observation vector e. The standard reflection formula 
is a vector relationship that satisfies SnelTs law of reflection, where the angle of 
incidence ai (FIG, 3) is equal to the angle of reflection ar (FIG. 3). 
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According to one embodiment of the invention, VPU 213 (FIG. 2) uses a 
modified version of the standard reflection formula to compute reflection 
vectors. For the point P on surface 305 located directly in front of observer 
310a (i.e., observation vector e intersects x-axis 320 at a right angle), the 
5 standard reflection formula is modified such that a reflection vector given by 
the modified reflection formula is equal to the normal vector n. That is, r' = n. 
Thus, the modified reflection projection method produces_the same result as 
the direct normal projection method when the point P on surface 305 is located 
directly in front of observer 3 10a. 

10 In order to modify the standard reflection formula, a pseudo-normal 

vector n' is defined that bisects the angle of incidence ai subtended by the 
observation vector e and the normal vector n. That is, n' = (e + n)/( | e + n | ) 
where | e + n | is the magnitude of e + n, and angle bi is equal to angle b r . 
When the pseudo-normal vector n' is substituted for the normal vector n in the 

15 standard reflection formula, the resultant modified reflection vector r' is equal 
to the normal vector n, since the modified reflection formula is based on the 
principle of Snell's law, where angle bi = angle b r . Thus, the modified reflection 
formula is expressed as r' = n = e - 2(e-n , )n'. 

A simplification of the modified reflection formula is straightforward. 

20 Assuming that point P is located at (x,y,z) = (0,0,z), then the unit normal vector 
n has components [nx, ny, nz] and the unit observation vector e has 
components [0,0,-1], where brackets [ ] are used to specify vector quantities. 
For example, [nx, ny, nz] is another way of writing vector n. Now, substituting 
the components of n and e into the expression for n', one obtains 

25 n' = (e + n)/( | e + n | ) = [nx, ny, nz-l]/(sqrt(nx 2 + ny 2 + (nz-1) 2 )). Expanding the 
argument of the square root in the denominator, one obtains nx 2 + ny 2 + (nz-1) 2 
= nx 2 + ny 2 + nz 2 + 1 - 2nz = 1 + 1 - 2nz = 2(1 - nz), since the normal vector n is 
a unit vector of magnitude one. If k = l/sqrt(2(l - nz)), then n' = k[nx, ny, nz- 

1]. 

30 Now, substituting n' into the modified reflection formula, one obtains 

r* = n = e - 2(e-n')n' = e - 2k 2 (e-[nx, ny, nz-l])[nx, ny, nz-1] = e - (e-[nx, ny, nz- 
l])[nx, ny, nz-l]/(l - nz). That is, r' = 6 - (e[nx, ny, nz-l])[nx, ny, nz-l]/(l - nz) 
for any given observation vector e. In other words, the modified reflection 
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formula is valid for any given observation vector e = [ex, ey, ez], and any point P 
on surface 305 with an associated unit normal vector n = [nx, ny, nz]. For 
example, if observer 310b views point P along an observation vector e", then 
VPU 213 uses the modified reflection formula to compute a reflection vector r", 
5 where r" = e" - (e"-[nx, ny, nz-l])[nx, ny, nz- 1]/(1 - nz). 

The modified reflection formula may be simplified further, and expressed 
by a more compact mathematical relationship. For example, if the unit 
observation vector e = [0,0,-1] is relabeled as a constant reference observation 
vector e 0 , then the modified reflection formula may be written as 

10 r' = e - (e*[nx, ny, nz-l])[nx, ny, nz-l]/(l - nz) = e - (e-(n+eo))(n+e 0 )/(l-nz). 

FIG. 4 is a flowchart of method steps for displaying an image of a 
reflective object based upon texture coordinates, according to one embodiment 
of the invention. In step 405, a user loads video software into memory 210 
(FIG. 2) via optical disc control unit 234 (FIG. 2), for example, and CPU 212 

15 (FIG. 2) executes the video software. The video software may be an interactive 
or non-interactive video, and in an exemplary embodiment of the invention, the 
video software is a video game. In step 410, CPU 212 generates rendering 
instructions for all reflective objects of a video frame. The rendering 

■ 

instructions may be generated in response to user input received via controller 
20 interface 220 (FIG. 2). In step 415, VPU 213 (FIG. 2) executes the rendering 
instructions using an illumination model, and generates reflection vectors for 
each reflective object of the video frame. For example, a reflection vector is 
generated for each vertex point P (FIG. 3) of surface 305 (FIG. 3). Step 415 is 
further discussed below in conjunction with FIG. 5. 
25 In step 420, VPU 213 transforms the reflection vectors associated with 

each object to texture coordinates. The transformation may be a reflection 
vector mapping method, or may be configured using other known methods in 
the art. In addition, VPU 213 may compute a texture map composed of the 
texture coordinates for each reflective object. Next, in step 425, VPU 213 sends 
30 the texture coordinates and/or texture maps to GPU 214. In step 430, GPU 
214 prepares an image of each reflective object for display on a display device 

# 

(not shown), based in part on the texture coordinates or texture map associated 
with each reflective object. GPU 214 may use other illumination terms 
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generated by VPU 213 or CPU 212 in conjunction with the texture coordinates 
to prepare each reflective object for display as an image. 

Next, in step 435, CPU 212 determines if execution of the video game has 
been terminated. If execution has not terminated, then the method continues 
5 with the next video frame at step 410. However, if in step 435, CPU 212 

ascertains that execution of the video game has terminated, then the method 
ends. 

FIG. 5 is a flowchart of method steps for generating reflection vectors for 
a reflective object, according to one embodiment of the invention. In step 505, 

10 VPU 213 (FIG. 2) selects a vertex point P (FIG. 3) on surface 305 (FIG. 3) of the 
reflective object. Next, in step 510, VPU 213 obtains vector components 
[nx,ny,nz] of a normal vector n (FIG. 3) to the surface 305 of the reflective object 
at the selected vertex point P. In one embodiment of the invention, the vector 
components are stored in registers (not shown) associated with CPU 212. In 

15 another embodiment of the invention, the vector components are stored in a 
memory (not shown) associated with VPU 213. 

In step 515, VPU 213 determines components [ex",ey",ez"] of the 
observation vector e" (FIG. 3), for example, directed from the point P to observer 
310b (FIG. 3). VPU 213 may compute the components [ex",ey",ez"] or may 

20 receive the components from VPU 213 memory (not shown) or CPU 212 

registers (not shown). Next, VPU 213 uses a modified reflection formula, the 
components (ex",ey",ez") of the observation vector e", and the vector 
components (nx,ny,nz) of the normal vector n to compute a reflection vector r" 
(FIG. 3), in step 520. In step 525, VPU 213 determines whether a reflection 

25 vector for each vertex point P associated with the reflective object has been 

computed. If a reflection vector for each vertex point P has not been computed, 
then in step 530, VPU 213 selects another vertex point P on surface 305 of the 
reflective object. The method then continues at step 510: However, if in step 
525, a reflection vector for each vertex point P has been computed, then the 

30 method ends. 

The invention has been explained above with reference to several 
embodiments. Other embodiments will be apparent to those skilled in the art 
in light of this disclosure. The present invention may readily be implemented 
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using configurations other than those described in the embodiments above. 
For example, the modified environment reflection projection method, according 
to the invention, may be executed in part or in whole by CPU 212, VPU 213, 
GPU 214, or a rendering engine (not shown). Or, for example, the modified 
5 environment reflection projection method may be implemented in parallel by a 
multiprocessor system. Additionally, the present invention may effectively be 
used in conjunction with systems other than those described in the 
embodiments above. Therefore, these and other variations upon the disclosed 
embodiments are intended to be covered by the present invention, which is 
10 limited only by the appended claims. 



* 
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WHAT IS CLAIMED IS: 

1 1. A method for environment mapping, comprising the steps of: 

2 determining a surface normal vector n at a point P on a surface of a 

3 reflective object; 

4 determining an observation vector e from the point P to an observer; 

5 and 

6 using a modified reflection formula to compute a reflection vector r 

7 based on the surface normal vector n and the observation vector 
S e, the modified reflection formula based on reflection about a 

9 pseudo-normal vector n' at the point P on the surface. 

1 2. The method of claim 1, wherein the pseudo-normal vector n' bisects an 

2 angle subtended by the surface normal vector n and a reference observation 

3 vector e 0 , the reference observation vector e 0 directed from the point P to the 

4 observer located directly in front of the point P. 

1 3 . The method of claim 1 , wherein the modified reflection formula is 

2 r = e - (e*[nx, ny, nz-l])[nx, ny, nz-l]/(l - nz), where nx, ny, and nz are the 

3 components of the surface normal vector n. 



1 4. The method of claim 1, wherein the point P on* the surface of the 

2 reflective object is located at a vertex of a polygon. 

■» 

1 5. The method of claim 1, further comprising the step of transforming the 

2 reflection vector r to texture coordinates (s,t). 

1 6. The method of claim 5, further comprising the step of rendering the 

2 reflective object based on the texture coordinates (s,t) associated with each 

3 point P on the surface of the reflective object. 
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1 7. An electronic-readable medium having embodied thereon a program, the 

2 program being executable by a machine to perform method steps for 

3 environment mapping, the method steps comprising: 

4 determining a surface normal vector n at a point P on a surface of a 

5 reflective object; 

6 determining an observation vector e from the point P to an observer; 

7 and 

8 using a modified reflection formula to compute a reflection vector r 

9 based on the surface normal vector n and the observation vector 

10 e, the modified reflection formula based on reflection about a 

11 pseudo-normal vector n' at the point P on the surface. 

i 

1 8. The electronic-readable medium of claim 7, wherein the pseudo- 

2 normal vector n r bisects an angle subtended by the surface normal vector n 

3 and a reference observation vector e 0 , the reference observation vector e 0 

4 directed from the point P to the observer located directly in front of the point 

5 P. 
1 

1 9. The electronic-readable medium of claim 7, wherein the modified 

2 reflection formula is r = e - (e-[nx, ny, nz-l])[nx, ny, nz-l]/(l - nz), where nx, 

3 ny, and nz are the components of the surface normal vector n. 

1 10. The electronic-readable medium of claim 7, wherein the point P on the 

2 surface of the reflective object is located at a vertex of a polygon. 

1 11. The electronic-readable medium of claim 7, further comprising the step of 

2 Ixansforming the reflection vector r to texture coordinates (s,t). 



1 
2 
3 



12. The electronic-readable medium of claim 1 1, further comprising the step 
of rendering the reflective object based. on the texture coordinates (s,t) 
associated with each point P on the surface of the reflective object. 



WO 03/032253 PCT/US02/32421 



1 13. A system for environment mapping of a reflective object, comprising: 



2 a memory configured to store a modified reflection model, the modified 

3 reflection model based on reflection about pseudo-normal vectors 

4 located at points on a surface of the reflective object; 

5 a vector processing unit configured to compute reflection vectors using 

6 the modified reflection model; and 

7 a graphics processing unit configured to render the reflective object in an 

8 image, the quality of the image dependent upon the computed 

9 reflection vectors. 



1 14. The system of claim 13, wherein the modified reflection formula is 

2 r = e - (e-[nx, ny, nz-l])[nx, ny, nz-l]/(l - nz), where nx, ny, and nz are 

3 components of a surface normal vector n at a point P on the reflective surface, 

4 and e is an observation vector directed from the point P to an observer. 

1 15. The system of claim 13, wherein a pseudo-normal vector n' bisects an 

2 angle subtended by a surface normal vector n at a point P on the reflective 

3 surface and a reference observation vector e 0 , the reference observation vector 

4 e 0 directed from the point P to an observer located directly in front of the point 

5 P. 

1 16. The system of claim 13, wherein the vector processing unit is further 

2 configured to process each computed reflection vector to generate texture 

3 coordinates (s,t). 

1 17. The system of claim 16, wherein the graphics processing unit uses the 

2 texture coordinates to render the reflective object as an image. 
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1 18. A system for environment mapping, comprising: 

2 means for determining a surface normal vector n at a point P on a 

3 surface of a reflective object; 

4 means for determining an observation vector e from the point P to an 

5 observer; and 

6 means for using a modified reflection formula to compute a reflection 

7 vector r based on the surface normal vector n and the 

8 observation vector e, the modified reflection formula based on 

9 reflection about a pseudo-normal vector n' at the point P on the 
10 surface. 
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(57) Abstract: A system and method for environment mapping determines a computer-generated object's reflective appearance, 
based on position and orientation of a camera with respect to the object's location. The present invention is implemented as a real- 
time environment mapping for polygon rendering (fig. 3), however, the scope of the invention covers other rendering schemes. 
According to one embodiment of the present invention, a vector processing unit [VPU] (213) uses a modified reflection formula 
to compute reflective properties of an object. The modified reflection formula is: r = e - (e*(n+eo))(n+eo)/(l-nz) = e - (e # ([nx, ny, 
nz-l])/(l - nz), where eo = [0,0,-1], and nx, ny, and nz are the components of the surface normal vector n. 
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